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EXECUTIVE SUMMARY 

The Test Readiness Report (TRR) is the second of two planning 
documents to be completed prior to a Test Readiness Review. The Test Plan is 
the first document, and it lays out high-level objectives, roles and 
responsibilities, and other preliminary information for early approval. The TRR 
documents that all necessary preparations for conducting the test have been 
completed. It builds on and supercedes the Test Plan and is the execution plan 
for conducting the runs for record. Signatures on the TRR constitute authority 
to begin testing 

This template will be tailored depending on the type of event. The section 
heading indicates paragraphs that are only appropriate for certain types of test 
venues. 

In the TRR Executive Summary, provide a summary of essential 
information regarding the testing/simulation event. Include high-level 
objectives, dates and location of the event and how the results will be 
used. Provide a summary to support a recommendation to proceed 
forward with the test based on the following outline: 

1. System test status and checkout performance 

2. Federation Object Model (FOM) status (M&S venues) 

3. Equipment and computer program configuration 

4. Test coordination 

5. Success criteria 

6. Go/No-Go criteria 

7. Recommendation for accreditation of federation (M&S venues) 

8. Recommendation to proceed with the test. 
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STYLE AND FORMATTING GUIDELINES 

This template has specific style types built into it to allow common 
formatting across Test Readiness Reports. Headings are defined as first order, 
second order, third order, and so on; or, as number one, number two, and 
number three. There should seldom be a number four heading. These heading 
styles are called Heading 1, Heading 2, Heading 3, and Heading 4. They are of 
Bookman Old Style font, are boldface, and not underlined. Numbering goes as 
1., 1.1, 1.1.1, etc. 

Figures use the style “Caption.” Tables use the style “Table Center.” 
Appendices use the style “Annex.” 

Updating Table of Contents, List of Figures, List of Tables, and List of 
Appendices is done using the following steps: 

a) Identify the table or list you wish to update and right-click inside it. 

b) Select “Update field.” 

c) If you want to update the table entries AND pages numbers, select 
“Update entire table.” If you want to just update page numbers, select 
“Update page numbers only.” 

In accordance wath JSSEO configuration management polity, the footer 
of the document should have the following format: 

WBS number_Test Readiness Report(Document Control Number)_Version 
Number_JSSEO_YYMMDD 
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1. INTRODUCTION 

1.1 Background 

Summarize significant historical data, outline key players for the test, 
approaches to testing, major focus areas, and capabilities of the 
testing/simulation process. Discuss the basis of approach for testing. 
Reference should be made to previous related tests, problems found during 
operational use, etc. Include topics such as: 

1. Dates of Significant Milestones 

2. Origin 

3. Process 

4. Timeframe and Priorities 

5. Location 

6 . Environment 

7. Provide a brief description of what system{s) is/are under test. 

1.2 Purpose of Test 

Succinctly state the top-level purpose of the test. Identify the customer 
for the test results. Describe the final product of the test (i.e., the deliverable) 
and how the customer will use it. 

1.3 Scope of Test 

Identify the top-level test objectives, hypotheses, and test description. 
Identify the participating organizations, test elements, and assessment 
constraints and limitations. 


Page 1-1 

7.2.7.2_TRR(04-016)_ 1.0Z_JSSEO_041210 
UNCLASSIFIED 



UNCLASSIFIED 


THIS PAGE INTENTIONALLY LEFT BLANK 


Page 1-2 

7.2.7.2_Standard Test Readiness Report(04-0016L1.0Z_JSSEO_04I210 

UNCLASSIFIED 



UNCLASSIFIED 


2. OVERALL TEST DESIGN 

2.1 Concept of Test Operations 

Provide an overview of the experiment(s) that will be conducted during 
the test. 

2.2 Brief Experiment Description 

Each experiment will have its own description that should follow the 
outline described in this Section. 

2.2.1 Experiment Objectives 

Provide details regarding the specific test. State the date, how many 
runs will be conducted, and overall objectives. Also provide specifics regarding 
the test data to be examined. 

Provide details regarding the number of runs to be conducted and how 
they will be conducted. Provide a sequence of events. For example: 

1. System baseline 

2. Time events 

3. Data registration events 

2.2.2 Experiment Hypothesis 

Briefly describe hypothesis to be provided or disproved in the experiment. 
Include any relevant background or specific information about experiment. 
Specify under what conditions issue occurs. 

2.2.3 Attributes and MOPS Measured 

Briefly describe the parameters or outputs that will be used to evaluate 
system performance. MOPs should be short definitive statements beginning 
with an action verb (e.g., “measure” or “calculate”). More detailed description 
of the MOPs should be provided in the Data Analysis Plan Appendix. 

2.2.4 Data Management and Success Criteria 

Summarize data and instrumentation requirements and data 
management strategy. A Data Management and Analysis Plan will be provided 
as an appendix to the Test Readiness Report. 
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For the data requirements listed, identify a process for determining that 
data has been properly collected. (Did the test go as planned? Was data 
collection successful? Is data quality sufficient for post-event analysis? More 
or supplemental data needed? EOIs identified and packaged for analysis? 
TORs collected? Media/tapes set for next op?) . 

2.2.5 Test Methodology 

Describe test procedures associated with the MOP to acquire the 
appropriate information to adequately answer the MOP. 

2.2.5.1 Baseline Experiment 

Describe how a baseline for Critical Experiments will be established. 

For example: "The first set of runs will support establishing a baseline for 
the E-2C SIAP performance. Two runs will be taken to ensure that the data 
between the two runs produces similar SIAP results and that the process is 
repeatable. SIAP attributes will be calculated for these runs and will be used 
as the standard bearer against which all parametric analysis will be compared. 
It is expected that both operator/analyst observations and the SIAP attributes 
will reflect a minimum of differences between the two runs. If repeatable 
baseline runs are not achieved, parametric runs will not be conducted until the 
cause for lack of repeatability is determined and fixed." 

2.2.6 Requisites 

Identify the operational context required to properly collect the data for 
the experiment. Include number of units required. 

2.2.7 Data Reduction and Analysis Method 

Identify the data reduction process, including tools used, how the data 
will be used and by whom, and how the data will be provided to analysis team. 
Lay out analysis process: Identify data reduction process (tools used, who is 
doing what with the data, how data is provided to analysis team); analysis 
method, including description of tools/algorithms for conducting analysis. 

2.2.8 Analysis Team 

List the analysis team lead and key team members. List units involved 
in the experiment and points of contact. 
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2.2.9 Reporting Schedule 

Provide the reporting schedule for the analysis to be conducted. Include 
any constraints or contingencies on delivering the report. 

2.3 Additional Experiments 

If the test includes multiple experiments, describe the first critical 
experiment in section 2.2, then add sections 2.3, 2.4,..., 2.n as necessary for 
each of n critical experiments. Follow the format of section 2.2 for these 
additional sections. 
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3. MODELING AND SIMULATION (M&S Venues) 


3.1 Federation Design 


This section provides a description of the test, including design and roles 
of each federate. Include an overview of the components, interfaces, systems' 
roles in the federation, how they are implemented, and any support elements 
(Figure 1). List each federate and document further detail for each. 
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Figure 1. Notional Federation Design 

3.2 Federate and Federation Component Roles 

Provide a functional description of the Federates that will be used during 
the event. 

3.2.1 Federate Name (e.g., E-2C Federate, ESTEL) 

Role in Federation: 

• State federate's role(s) in the federation. 
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• For example: Simulates E-2C APS-145 radar, IFF 
interrogator/transponder, and navigational systems. 

Constraints / Limitations 

• State federate's constraints/limitations. 

Implementation: 

• State federate's implementation. 

• For example: AN/APS-145 Radar is simulated using RISS 

Federation Verification, Validation, and Accreditation (W&A): 

• State pertinent W&A information. 

3.2.2 Support Federates 

Identify and describe support federates required for the event. For 
example: 

Test Control 

• Adapted from Navy Infrastructure (NI) effort 

• Provides federation start/stop and monitoring. 

hloResults® Version 2.0 

• Commercial product to collect data in federation and play back data. 

3.2.3 Supporting Tools 

Identify and describe supporting tools that are required for the event. For 
example: 

Command, Control, Communication, and Intelligence (C3I) Engineering and 
Evaluation Snstem (CEES) 

• Interoperability tool developed by Redondo Systems, Inc. 

• Monitors and collects TADIL J and DIS truth data. 

Joint Analusis Disvlau Environment (JADE) 

• Three-dimensional quick-look tool during runs. 

• Monitors and collects TADIL J and HLA truth data. 

• Post-mission three dimensional (3D) replay capability 
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Tactical Office (TACO) 

• Three-dimensional quick-look tool during test runs. 

• Monitors and collects ECS, ICC, TADIL J, and DIS truth data. 

• Post-mission 2D replay capability. 

Performance Evaluation Tool (PET) 

• Metrics evaluation tool developed by NSWC Corona. 

• Incorporates ECS, ICC, TADIL J, and HLA truth data. 

• Post-mission 2D replay capability. 

• Seamless interoperation with ARCTIC. 

Automatic Reconstruction and Correlation Tool for Interoperabam 

Characterization (ARCTIC) 

• Performs Automatic Truth to System track matching. 

• Seamless interoperation with PET. 

• Flexible/tailorable to all types of system data. 

Details on the federates and federation, including the data exchange 
among the federates as specified in the federation object model (FOM) and the 
federation agreements can be found in the APPENDIX H: Federation 
Description. The equipment and computer program configurations of the 
federation are found in APPENDIX D: Test Configuration. 

3.3 Verification, Validation, and Accreditation (W&A) Process 

W&A are required to determine that a simulation or federation of 
simulations is appropriate to use for a particular test objective. Models and 
simulations must be accredited for their intended use. This is particularly 
important if a new version was required to be built to meet the test objectives. 
Additionally, step 5 of the FEDEP process involves integrating and testing the 
federation. Within that step, JSSEO has developed a verification, validation, 
and accreditation process that will be applied to this test. DoDI 5000.61 
provides the following definitions: 

Verification: the process of determining that a model implementation and 
its associated data accurately represent the developer’s conceptual 
description and specifications. Verification answers the question, “Did I 
build the thing right?” 

Validation: the process of determining the degree to which a model and 
its associated data are an accurate representation of the real world from 
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the perspective of the intended uses of the model. Validation answers 
the question, “Did I build the right thing?” 

Accreditation: the official certification (by the user) that a model, 
simulation, or federation of models and simulations and its associated 
data are acceptable for use for a specific purpose. Accreditation answers 
the question, “Does it suit my needs?” 

The JSSEO Technical Report on M&S W&A (2003-006) discusses how 
JSSEO is charged with providing recommendations to decision authorities in 
the Office of Secretaiy of Defense (OSD) and Joint Staff on how to achieve 
SIAP-related requirements across all Services and Agencies. These 
recommendations must be reviewed by the affected Services and Agencies in 
order to achieve consensus on their implementation. 


An accreditation decision is ultimately part of the overall risk assessment 
and analysis process used by JSSEO. The V&V activities supporting the 
accreditation decision help answer the questions; what is the likelihood (risk) 
that the data resulting from an M&S based analysis does not rejlect real-world 
systems or conditions, and what is the impact to the analysis? Therefore, the 
V&V activities should focus on assessing, to a high level of confidence, the 
suitability of M&S to produce the data necessary to meet a specified objective, 
in support of JSSEO decisions. 

Describe how the test team will meet the W&A needs. 

Describe the W&A process and procedures and the use of V&V Plans 
and V&V Reports. 

Figure 2 shows the SLAP W&A process. If the Test Readiness Report 
deviates from this process, provide a new process diagram. 
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Figure 2. JSSEO W&A Process 
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4. TEST SCHEDULE 

Present the overall testing schedule (Figure 3), in accordance with the 
project schedule. Show the schedule of events in list or timeline format (Gantt 
chart). Include pre- and post-test requirements. 
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Figure 3. Notional Schedule 
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5. TEST MANAGEMENT AND ORGANIZATION 
5.1 Roles and Responsibilities 

Provide an organizational diagram for conducting the test. Figure 4 
provides a notional organization of the event. The specific roles and 
responsibilities will be discussed for each organization. 


Resource Providers 

-JTAMDO 

-JSSEO 

-Services 

-JDEP 

-others 



JSSEO 


- Test Director 


- Customer 


- Event Coordinator 


- TP and DMAP 


- SAT SME Support 


- CRS Team 


- VV&A Oversight 


JNiC 

- Application Area Manager 

- MDA Coordination 

- AMD Data Repository 


'- y -- 


JITC , ■ 1 

Corona 

- Infrastructure/T'- ’ ^ic> *3. 

- Analysis Support 




I 








kilt t Martin NSCC PTC-1 
“s'Site Director/ Test Conductor/ 
Data Collection Manager 


SME Support Staff 
-Platfoim Analysts 
-Technical 
-Data Collection 
-Critical Experiment 
-Corona Analysts 
-FOM 

-Site Security 
-V&V 


Figure 4. Notional organization of an event 

5.1.1 Customer Name (e.g., JSSEO) 

The customer is the primaiy user of the test results. 

The customer: 

- Has primaiy responsibility for marshalling funding resources 

- Describes the expected level of support for the event 

- Provides some resources for the event 

- Coordinates the event 
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- Oversees overall planning, conduct, and analysis of event 
Coordinates Test Plan and Test Readiness Report development and 
data management and analysis plan 

- Provides guidance on critical experiments via subject matter 
experts 

Develops the CRS excursion 

- Provides the V&V process 

- Has final accreditation authority for the event. 

5.1.2 Test Sponsor Name (e.g., Joint Theater Air and Missile Defense 
Organization (JTAMDO)) 


The Test Sponsor is a resource provider and endorses the scope and 
goals of a project and represents the test throughout the management process. 
The Test Sponsor exercises approval authority over Test 
Obj ectives/Plans / Results. 

5.1.3 AppUcation Area Manager (e.g., Joint National Integration Center 
(JNIC)) 

The JTAMD Application Area Manager provides technical environment 
support services, maintains visibility over a family of systems, and oversees 
test requirements. 

The JTAMD Application Area Manager: 

Reviews, evaluates test objectives, plans, analyses, and reports 
- Participates in event planning, execution, data collection, and 
analysis 

Provides insight for other test activities and applications to the 
broader testing community 


5.1.4 Infrastructure/Technical Manager (e.g., Joint Interoperability 
Command (JITC)) 

The Infrastructure/Technical Manager is responsible for developing the 
federation. 

The Infrastructure/Technical Manager: 

- Develops and executes a V&V plan for the Utility Player. 

- Is the Configuration Manager with the responsibility for ensuring 
that the FOM is configured properly and computer program 
versions used are documented 

Coordinates and maintains the Federation Agreements and 
coordinates FOM changes 


7.2.7.2_Standard Test 
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- Will provide technical assistance, if requested, to issues involving 
HLA federate design or the RTI. 


5.1.5 Participating Service(s) (e.g.. Lower Tier Project Office/Software 
Engineering Directorate (LTPO/SED)) 

Identify the participating Service(s) for this event. 

Participating Services will: 

- Develop test procedures for conducting experiments 

- Conduct V&V of their federate components in the test 
Execute test runs 

- Provide Subject Matter Experts to ensure test objectives are 
properly addressed. 

5.1.6 Supporting Agencies (e.g., Naval Surface Warfare Center (NSWC) 
Corona) 

Identify roles and responsibilities for Supporting Agencies. 

Supporting Agencies; 

- Ensure that the test{s) accurately capture program attributes 
Provide on-site analysis, as necessary 

5.1.7 SIAP Analysis Team (SAT): Executive Steering Group (ESG) and 
Other Test Representatives 

Identify the SAT ESG members associated wdth the subject test and their 
intended roles and responsibilities. It should include statements regarding 
whether the SAT ESG is expected to provide the resources necessary to plan, 
execute and analyze an event. It is the responsibility of SAT members to 
ensure the right tools are brought to collect necessary data and perform on-site 
analysis. 

5.1.8 JSSEO Common Reference Scenarios (CRS) Team 

Identify the CRS team that will be responsible for developing CRS 
excursions that reflect the needs of the event. 

The SIAP CRS Team will: 

- Develop the scenario with elements and formats consistent with 
the FOM 

- Ensure the scenario contains the appropriate requisites to conduct 
experiments 
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- Provide data required to conduct test. 


5.2 On-Site Organization 


Clearly outline the management roles of on-site activity. Identify one 
overall leader and assistant managers (one for SAT, one for critical experiments 
and may need one for another area of testing). 

Roles for SAT include developing Distinguished Visitors (DV) stoiyboards 
before heading on-site. Members of the SAT should be prepared to discuss 
mission monitoring of the display tools to any of the Distinguished Visitors 
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6. TEST READINESS REVIEW PREPARATIONS 

The purpose of the Test Readiness Review is to present results and 
status of the preparations for the test to the accreditation authority or 
leadership (whichever the case) to enable a decision to be reached to proceed 
with the test. Test Readiness Report approval is the desired outcome of the Test 
Readiness Review. The Test Readiness Review should have the following 
information included for discussion; 

1. System test status and checkout performance 

2. FOM status (M&S venues) 

3. Equipment and computer program configuration 

4. Test objective(s) and procedure review 

5. Test coordination 

6. Security 

7. Success criteria 

8. Go/No-Go criteria 

9. Real-time data requirements to include format, algorithms, and data 
definitions 

10. Quick-look data requirements to include format, algorithms, and 
data definitions (if available) 

11. Final data requirements to include format, algorithms, and data 
definitions 

12. Recommendation for accreditation of federation (M&S venues) 

13. Recommendation to proceed with the test. 

6.1 Tasks Accomplished 

The Test Readiness Review should include results from dry-run testing, 
including the data required to justify V&V results (M&S venues). 
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7. TEST EXECUTION 

7.1 Pre-Test Briefing 

The purpose of the Pre-Test briefing is to ensure that all participants 
understand the test procedures, their individual roles and responsibilities, and 
the test Go/No-Go criteria. The Pre-Test briefing is delivered by the Test 
Conductor and takes place every day prior to starting test runs. All members of 
the test team, including test component operators as well as any on-site test 
support staff, should be in attendance. The attendees make a recommendation 
to the Test Conductor on whether Go/No-Go criteria have been met, but the 
Test Director makes the final determination. The pre-test brief should have the 
following information included for discussion; 

- System test status and checkout performance 

- FOM status (M&S venues) 

- Equipment and computer program configuration 

- Test objective and procedure review 

- Test coordination 

- Security 

- Success criteria 

- Go/No-Go criteria 

- Real-time data requirements to include format, algorithms, and data 
definitions. 

- Quick-look data requirements to include format, algorithms, and data 
definitions 

- Final data requirements to include format, algorithms, and data 
definitions 

7.2 Test Execution and Data Collection 

Provide any Instructions about executing the test such as following the 
test procedures and run matrix described in the appendices. Describe any 
special data collection tools or activities required for the test. 

7.3 Daily Test Schedule 

Provide a daily test schedule (Figure 5). 
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Figure 5. Notional Test Schedule 


7.4 Data Analysis 

Identify who will compute the data and how the results will be presented. 
Figures K-2, K-3, and K-4 of the Data Analysis Appendix K give notional 
examples of these results. 

7.5 Test Observation Reports (TORs) 


State how documents that capture perceived anomalies or incidents that 
require further analysis will be utilized within the framework of the test. 
Discuss the TOR adjudication process. For example. "Results of relevant TORs 
will be incorporated in the 'Lessons Learned' portion of the E-2C Pilot Report. 
An example TOR form is provided in APPENDIX L." Detail a contingency plan 
that has TOR database work-arounds in place. 

7.6 Post Test Briefing 

A briefing by the Test Conductor should be provided to the test team 
following each day’s test runs to highlight lessons learned and any other 
relevant issues. 
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8. TEST REPORTING 

8.1 Quick-Look Report 

Identify the organization(s) responsible for producing and/or reviewing 
the quick look report and the timeline by which the report will be submitted. 
Also, identify topics that should be covered. Topics the Quick-Look will cover 
include: "Evaluation of findings from a management perspective, significant 
test results, and preliminary conclusions." 

8.2 Technical Report Outline 

In this section, identify organization(s) responsible for producing and/or 
reviewing the final report. Set the timeline for submission. Establish the 
coordination process, through final approval authority. State expected format 
for the final report. For example: "A technical report will be generated within 
90 days following the E-2C JDEP event. Generating the report will be a 
collaborative effort. Final signature will be provided by JSSEO, JTAMDO, JNIC, 
JITC, and E-2C." 

The final report will include a description of the experiment as it was 
actually conducted (parametric runs) with enough detail such that the test can 
be repeated, a summary of the SIAP attributes results, discussion of the results 
including root cause, and recommendations for improvement. A typical 
technical report for an event will have the outline shown in Table 1. 

Table 1. Standard Results Technical Report Outline 

EXECUTIVE SUMMARY 

INTRODUCTION 

Purpose/Intent 

Background 

Overall Test Objectives 

Assessment Constraints and Limitations 

ASSESSMENT RESULTS AND ANALYSIS 

General 

Analysis Objectives 
Objective I. 

Objective 2. 

Objective 3. 

Analysis Products 
On-site Activity 

On-Site Objectives 
Organizational Analysis Support 
Approach/Methodology 
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Data Collection 
Test Procedures 

Test Observation Report (TOR) Process 

Data Availability Matrix 

Results 

Post-Event Analysis 

Post-Event Objectives 

Approach /Methodology 

TSPI Discussion 

Track Matching Process 

PET Description and Processing 

Prioritized TORs and Events of Interest (EOIs) 

Critical Experiments 
Additional Analytical Issues 
LESSONS LEARNED 

CONCLUSIONS AND RECOMMENDATIONS 
REFERENCES 

APPENDICES: ACRONYMS, FORMAL ANALYSIS REPORTS 
INSTRUMENTATION, EXTENSIVE DATA (TABLES), MATHEMATICAL METHODS 
POINTS OF CONTACT 

Table 2 gives the schedule for the reporting process. 


Table 2. Notional Reporting Timeline 


Description 

Responsible Party(ies) 

Date 

Quick-look report 

Insert OPR: (e.g., 
ESTEL/Corona) 

30 days 

Review of Final Results 

Insert OPR: (e.g., SAT: 
JDEP representatives) 

45 days 

Review and comment 

Insert OPR: (e.g., JDEP 
Protect Lead, ESG) 

60 days 

Final Techniccil Report 
signed 

Insert OPR: (e.g., SIAP, 
JTAMDO, JNIC, JITC, and 
E-2C) 

90 days 


8.2.1 Summary and lessons Learned 

Identify lessons learned from the event, including issues with logistics, 
planning, execution, and analysis. Indicate how and by whom relevant TORs 
will be reviewed for candidacy into the SLAP Lessons Learned Knowledge Base 
(LLKB). Lessons Learned in the LLKB are generally separated into two 
categories, operational lessons and programmatic lessons. Operational lessons 
encompass any observed interoperability issues or events of interest noted 
while running the test. Programmatic lessons include any issues that deal 
with the planning, management and coordination of executing the test. 
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8.2.2 Unresolved Issues 

Analysis results will be documented in the final report in the context that 
the issue is either understood and recommendation is provided, the issue is 
not understood and needs additional time and resources to isolate, or that the 
issue is not problematic and is dropped. 

Indicate how issues requiring additional time will be addressed and how 
the responsible parties will resolve them. 

Interoperability issues will be discussed via phone, e-mail, or secure 
telephone unit (STU). The objective will be to isolate interoperability issues as 
far as possible in a distributed environment so as to avoid lengthy periods of 
co-located analysis. 
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9. REFERENCES 

List all relevant references to the document. 
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(2001, March). U.S. Joint Forces Command. 

Combat Identification Capstone Requirements Document (CID CRD), (2001) U.S. 
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Metrics Implementation, (2001, October). Arlington, VA: JSSEO. 

SIAP Stcmdard Data Management and Analysis Plan, Version I.l, (2002, July). 
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SLAP Common Reference Scenario Technical Report, Version 1.1, (2002, July). 
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APPENDIX A: ACRONYMS 


List all acronyms in the document. A set of frequently used acronyms is 
provided here and should be tailored for the Test Readiness Report. 


A 

AA 

ABT 

ACM/ACS 

AEW 

AGC 

ARCTIC 

ASCII 


Ambiguity 

Accreditation Authority 
Air-Breathing Threat 

Automatic Channel Monitoring/Automatic Channel Select 
Airborne Early Warning 
Automatic Gain Control 

Automated Reconstruction and Correlation Tool for 
Interoperability Characterization 

American Standard Code For Information Interchange 


C 

CCD 

CD 

CEC 

CID CRD 

CNA 

COTS 

CRD 

CRS 

CRSD 


Completeness (SLAP attribute) 

Common Carrier Device 
Compact Disk 

Cooperative Engagement Capability 

Combat Identification Capstone Requirements Document 

Center for Naval Analyses 

Commercial off the Shelf 

Capstone Requirements Document 

Common Reference Scenario 

Common Reference Scenario Driver 


DDM 

DEP 

DIS 

DISN 

DM 

DMAP 

DoDI 

DPCA 

DPG 

DR 

DX 

DX/DR 


Data Distribution Manager 
Distributed Engineering Plant 
Distributed Interactive Simulation 
Defense Information Services Network 
Data Manager 

Data Management and Analysis Plan 
Department of Defense Instruction 
Displaced Phase Center Array 
Defense Planning Guidance 
Data Recording/Data Reduction 
Data Extraction 

Data Extraction/Data Recording 


ESC /AW Electronic Systems Center (previously referred to as MASC) 

ESG Executive Steering Group 

ESTEL E-2C Systems Test and Evaluation Laboratoiy 


FOM Federation Object Model 

FoS Family of Systems 
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FTP 

GII 

GPS 

GRU 

GTE 

HLA 

HWIL 

LADS 

LAW 

ICC 

ICD 

ID 

IFF 

JCoCaC 

JDEP 

JIADS 

JITC 

JNIC 

JSSEO 

JTAMDO 

JTIDS 

KPP 

M&S 

MDA 

MIL-STD 

MOE 

MOP 

MS 

MSD 

NAVAIR 

NSWC 

OSD 

PC 

PET 

PO 


File Transfer Protocol 
Group II 

Global Positioning System 
Gridlock Reference Unit 
Gateway Terminal Emulator 

High-Level Architecture 
Hardware in the Loop 

Integrated Air Defense System 

In Accordance With 

Information and Coordination Central 

Interface Control Document 

Identification 

Identification Friend or Foe 

Joint Council of Colonels and Captains 

Joint Distributed Engineering Plant 

Joint Integrated Air Defense System 

Joint Interoperability Test Command 

Joint National Interoperability Center 

Joint Single Integrated Air Picture System Engineering 

Organization 

Joint Air and Missile Defense Organization 
Joint Tactical Information Distribution System 

Key Performance Parameter 

Modeling and Simulation 
Missile Defense Association 
Military Standard 
Measure of Effectiveness 
Measure of Performance 
Microsoft 

Modeling and Simulation Developer 
Navy Air 

Naval Surface Warfare Center 

Office of the Secretary of Defense 

Personal Computer 
Performance Evaluation Tool 
Program Office 
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POC 

Point of Contact 

PPLI 

Precise Participant Location and Identification 

PU 

Participating Unit 

R2 

Reporting Responsibility 

RISS 

Radar IFF Simulation System 

RTI 

Runtime Infrastructure 

SAT 

Single Integrated Air Picture Analysis Team 

SE 

System Engineer 

SLAP 

Single Integrated Air Picture 

SIF 

Selective Identification Feature 

Sim/Stim 

Simulation/ Stimulation 

SIPRNet 

Secret Internet Protocol Router Network 

SME 

Subject Matter Expert 

SoS 

System of Systems 

SPC 

Special Programs Center 

SWIL 

Software in the Loop 

STU 

Secure Telephone Unit 

TACCAR 

Time Averaged Clutter Coherent Airborne Radar 

TADIL 

Tactical Digital Information Link 

TAMD 

Theater Air and Missile Defense 

TAMD CRD 

Theater Air and Missile Defense Capstone Requirements 
Document 

TD 

Test Director or Tactical Driver 

TDDS 

TRAP Data Dissemination System 

TF 

Task Force 

TIAC 

Theater Air and Missile Defense Interoperability Assessment 
Capability 

TIBS 

Tactical Information Broadcast System 

TIM 

Terminal Input Message 

TO 

Test Objective 

TOM 

Terminal Output Message 

TOR 

Test Observation Report 

TPWG 

Test Plan Working Group 

TQ 

Track Quality 

TRAP 

Tactical Related Applications 

TSIU 

Tactical System Interface Unit 

W&A 

Verification, Validation, and Accreditation 

WAM 

Warfare Assessment Model 

WG 

Working Group 

WST 

Weapons Systems Trainer 
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2D 2 Dimensional 

3D 3 Dimensional 
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APPENDIX B: SIAP METRICS 

JSSEO developed a set of attributes (JSSEO Technical Report 2003-029) 
derived from TAMD and CID CRD key performance parameters. The Test 
Readiness Report should describe in this appendix any information that 
impacts the calculation of the SIAP attributes and any measures of 
performance. All JSSEO tests should include a SIAP attributes calculation. 

Any caveats, limitations, or changes from the ordinary to compute them should 
be mentioned here. For reference, the qualitative definitions of the SIAP 
attributes are provided as follows: 

Completeness: The measure of the portion of true air objects that 
are included in the SIAP. The air picture is complete when all 
objects are detected, tracked and reported. 

Clarity: The measure of the portion of the SIAP that contains 
ambiguous tracks and/or spurious tracks. The air picture is clear 
when it does not include ambiguous or spurious tracks. 

Continuity: The measure of how accurately the SIAP maintains 
track numbers over time. The air picture is continuous when the 
track number assigned to an object does not change. 

Kinematic Accuracy: The measure of how accurately the TAMD 
Family of Systems (FoS) reports track position and velocity. The 
air picture is kinematically accurate when the position and velocity 
of each assigned track agree with the position and velocity of the 
associated object. 

ID Completeness: The measure of the portion of tracked objects 
that are in an identified state. The ID is complete when all tracked 
objects are in an identified state. 

ID Correctness: The measure of the portion of tracked objects that 
are in the correct ID state. The ID is correct when all tracked 
objects are in the correct ID state. 

ID Clarity: The measure of the portion of tracked objects that are 
unambiguously identified. The ID is clear if no tracked object is in 
the ambiguous ID state. 

Commonality: The measure of consistency of the air picture held 
by TAMD FoS participants. The air picture is common when the 
assigned tracks held by each participant have the same track 
number, position, and ID. 
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The actual attribute computations will be automated through the use of 
the Performance Evaluation Tool (PET), into which the algorithms for the SIAP 
attributes have been encoded. 
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APPENDIX C: FEDERATION DEVELOPMENT PROCESS (M&S VENUES) 

The development of the federation designed to support this test follows 
the seven-step FEDEP process, which is now an IEEE standard process. This 
process provides the framework for the action plan and development schedule 
(Figure C-1). The steps in this process are shown in Figure C-1. 



Figure C-1. Federation development and execution process 


Step 1. Define Federation Objectives 

The first step of this process is to clearly define the federation objectives. 
This is key because all subsequent steps build on the objectives. 

Step 2. Perform Conceptual Analysis 

The next step is to define characteristics of federates and the federation 
needed to address issues. The federation requirements drive the selection of 
federates and the W&A of the federation. This step requires active 
participation of the subject matter experts and the system owners/proponents 
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because it is dependent on a sound understanding of the problem area, the 
substantive issues to be addressed in the test, and requirements for selection 
of the representations to meet the needs of the test. 

Step 3. Design Federation 

The next step is to identify specific federates, develop the Federation 
Object Model (FOM) for the federation, define federation CONOPS, and 
delineate federate upgrades to support the federation. The federation design 
reflects the decision of how to satisfy the federation requirements with specific 
federates, scenarios and data exchanges. At this stage it is almost always 
necessary to return to steps 1 and 2. It may be necessary to review the 
objectives for clarity and return to the conceptual analysis with more detail to 
ensure the requirements for the federation are well articulated and understood, 
and that the federation can be designed to meet the needs of the user. 

Step 4. Develop Federation 

Next, federate owners implement support for the FOM and 
enhancements in federates as needed and test individual federates. 

Step 5. Plan, Integrate, and Test Federation 

Incremental testing of federation capabilities and sets of federates is 
completed to prepare for the federation execution to support the test. 

Step 6. Execute Federation and Prepare Outputs 

The test is then conducted using the federation following the test process 
and procedures. 

Step 7. Analyze Data and Evaluate Results 

The final step is to conduct the data analysis, evaluate results, and 
produce the final report. 
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APPENDIX D: VALIDATION AND VERIFICATION (V&V) PLANS (M&S 

VENUES) 

As described in Section 2.3, the W&A process includes development of a 
V&V plan for the federates and the federation itself. Table D -1 identifies those 
federates requiring a V&V plan and the corresponding lead for each plan. 

Table D-2 gives a schedule of the W&A process for this test. 

Table D-1 Federates Requiring V&V Plan 


Federate requiring V&V 
Plan 

Responsible Party(ies) 

Primary Secondary 

Overall Federation 

Utility Player 

- PATRIOT Sim 
Interface 

- CRS-D 

- Tools (TIAC, . 

GEES, TACO] 

Utility Player / 

- GTE 1553 

- DLS 

- TIAC/HLA 

AV/T.-----:-. %. 

1 - \ ^ 

^ V-> 

Z - ^ 1 s 

4- 

^ < iiidaty Responsible 

** b Party 

M 

^ 1 lary Responsible Party 

Secondary Responsible 
Party 

PATRIOT Sim Interface 

- GTE X.25 

- FMS-D 

Primary Responsible Party 

Secondary Responsible 
Party 

CRS-D 
- CRS 

Primary Responsible Party 

Secondary Responsible 
Party 


Table D-2 V&V Schedule 


Date 

Action 

10 Mar 03 

All V&V plans delivered to M&S lead 

10-14 Mar 03 

V&V Activity team* review -^1' T 4 ' -- M&S lead provides 

recommendatlor-t ^ 7 v ^ L -al/disapproval of plans. 

19 Mar 03 

Status upd +< : i 1 1 \ ' including preliminary V&V 

reports. ^ 1, a ^ ^ 

7 Apr 03 

Telecon folio - report. M&S lead 

provides recc b y "1- prior to TRR to accredit or not 

accredit. % % ..m ^ 

9 Apr 03 

Test Readiness :t<_ view and accreditation. 


*V&V Action team: The W&A Action Team is an ad hoc team of SMEs, 
Model/Tool developers/experts, Service representatives and other specialists. It 
will normally be established as part of the Test Plan Working Group. Provide 
team members and representatives from each organization and identify their 
associated organizations. 
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The outline of the V&V Plan is specified in TR 2003-006, and is included 

below. 

1) M&S Requirements and Acceptance criteria. To determine the M&S 
requirements, a good understanding of the objectives and hypotheses is 
necessary. 

2 ) Capabilities/Limitations/Assumptions 

3) V&V Methods 

4) Data Certification 

5) M&S Development Methodology 

6) Configuration Management Plan 

As each V&V plan gets executed, the lead will indicate on the checklist of 
Table D-3 the V&V completion date and initial next to it. This checklist should 
be completed and provided at the Test Readiness Review. 


Table D-3. V&V Checklist 


Federate 

V&V Completion 
Date 

Initial 

Utility Player 

Responsible Parly 

— 

PATRIOT Sim/Stim 

Respou'-""’ 1 ’ ' 

CRS Driver 

F - - a ^ i 1 i 

Overall Federation 
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APPENDIX E: VALIDATION AND VERIFICATION (V&Vl REPORTS (M&S 

VENUES) 

The V&V Reports shall be included here as an appendix. The V&V Report 
documents the execution of the V&V Plan. 

Recommended outline: 

1. V&V Report 

1.1 Test objectives 

1.2 Accreditation Goals 

1.3 Accreditation process (Reference Accreditation Data from V&V runs 
and section 3 comparison table, and section 4 list of working group members. 

2. V&V Assessment Report 

2.1 Summary of Capabilities and Limitations (Based on V&V results and 
Differences Table in Section 3. 


3. Table 


Requirements 

Acceptance 

Criteria 

V&V Plan Test 
Result 

Difference 

between 

Acceptance 

Criteria and 
V&V Plan Test 
Results 












4. SME POC information (Test Plan Working Group) 

5. Recommendation to use or not to use federate in proposed test. 
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APPENDIX F: TEST CONFIGURATION 

Test Description 

Provide a brief test description that includes what the test entails. 

For example, "The E-2C pilot event will test the simulation and 
stimulation of the E-2C as biases are Introduced into its sensor, which, in this 
case, is its mission computer." 

Networks 

Internal Network 

Provide a description of the internal network for the test setup. 

For example, "The internal ESTEL network connections are shown in 
Figure F-1." 


S 

o 



tility player GTE 


Utility Player (UP) 


UP DISmUA GW 


Network 

Simulation Support 
E-2C 

Utility Player 
Timing Source 


IP Addresses 192.168.0.xxx 


Figure F-1. Example internal network diagram 
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External Network 


Provide a description of external network (connectivity, tools, etc). 

Seciirity 

Identify the classification levels for the testing facility, systems, and data 
produced during test. Also identify the security point of contact for the event 
and provide voice and e-mail contact information. 

"For all systems coming into the E-2C Systems Test and Evaluation 
Laboratoiy (ESTEL), the following items must be submitted two weeks prior to 
the install/integration date." 

Test Facilities 


Provide details about the test facility (i.e., background, location, etc.), the 
organizational structure, the facility function and the facility mission. 

For example, "ESTEL provides comprehensive test and evaluation of 
Airborne Early Warning (AEW) mission systems. ESTEL personnel comprise 
two distinct groups." 

HWIL/M&S Setup 

Provide a detailed description of the test setup. 

For example, "The Lab is composed of two tactical benches each 
supported by a mission system simulation suite, a mission playback system, 
and data reduction and analysis systems." 
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Radar video and trigger 
Antenna synchro 


Antenna azimuth 
Target data 
Control msgs 
Simulated CP->DP msgs 


Radar Hpts 


E-2C Position 
IFF Response 


E2C Position; 
IFF Interrogatif 
E-2C IFF Resp 


Target Position 
Target IFF 
Sim Management 


1553/J-Messages 





Nav Rpts 

% 

Radar Rpts 


Nav Rpts 

1 * . 

GPS Time 


Figure F-2. Example simulation suite data flow diagram 


Provide further details about the test facility, including connectivity/data 
link capabilities, processing elements, and associated parameters. 
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APPENDIX G: DETAILED SCENARIO DESCRIPTION 


General 

Provide details about the excursion scenario selection for the test. 

For example, "The excursion scenario is derived from the SlAP Common 
Reference Scenario (CRS) NEA III 2003 V2.0 scenario vignette. This scenario is 
30 minutes long, taken from 21:17-21:24 Zulu." 

Excursion Scenario Selection 

Discuss planning efforts that ensure the SlAP CRS elements meet the 
needs of the testing federate. 

For example, "The SLAP CRS team met with members of the E-2C testing 
team through JDEP Test Plan Working Group meetings and teleconferences to 
verify the requirements for the excursion scenario to meet test objectives. As a 
result, a 30-minute window was extracted from the CRS NEA III 2003 V2.0 
scenario vignette as the proposed excursion (CRS JDEP E-2C Excursion VI.2) 
to support this event. The selected excursion scenario was presented to and 
accepted by the E-2C testing team. The following sections describe the agreed- 
upon characteristics of the scenario and the E-2C-specified requirements that 
the scenario should provide." 

Characteristics 

Provide the scenario characteristics. For example, 

• Earth-Centered Inertial (ECI) coordinate frame 

• lOHz update rate 

• 3 degrees of freedom with orientation 

• WGS 84 J4 Oblate earth model 

• DTED included 

• EADSIM implementation files provided 

For example, "The CRS excursion offers a target-rich environment (both Red 
and Blue) for the purposes of examining SLAP issues and concerns. Although 
the scenario is not tactically correct, it meets the requirements of the test 
objectives of this event." 

Threat Order of Battle 

Provide a brief description of the threat order of battle. 
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Friendly Order of Battle 

Provide a brief description of the friendly order of battle. 

Scenario Requirements (Criteria) 

Provide the scenario requirements that meet the needs of the critical 
experiments as well as the needs of the system being tested. 

For example, "The scenario requirements for conducting the SLAP E-2C 
HWIL JDEP event include: 

• Operationally credible simulation environment 

• Sufficient threat aircraft in E-2C coverage area" 
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APPENDEC H: DETAILED TEST PROCEDURES 

This appendix contains the run matrix (for simulation tests) and detailed 
test procedures for the event. The run matrix is provided in Table H-1. If an 
alternative test matrix is devised, provide in Table H-2 and indicate under what 
conditions the alternative run matrix will be run (e.g., if time permits and the 
testing facility can meet the parameter sets). 

Run Matrix 

Table H-1. Run Matrix 


Run 

Test Objective 

Parameter Being 
Set 

Value 

Responsible 

Party 

1 

2 

3 

Establish E-2C baseline 
performance for Time 
S 3 mch., Navigation & 
Sensor Registration. The 
E-2C shall operate using 
standard operating 
procedures. 

EstabUsb E-.r.-fJ _ 

performi. 4 ‘ 'V" ? 

Synch., N -’c 

Sensor RCj t. iC 

E-2C shall , 1 

standard oi . \ % 

procedures. t 

Determine \ % 
time delta betiseeti Link- 
16 participants impacts 
either the weapon system 
or SLAP performance. 

Verify data extraction 

"DX" for CTR, NAV and 

DR analysis under 
normal procedures wip- ^ 
nomintd bias- . I 

%. 4 % i 

^ - X \ \ 1 

- 1 % f- 1 t 

i %■ %, \ \ 

% 


msert a time delta 

between the Utility 
player and E-2C mission 
computers. 

Recommended time delta 
is 1 hr., 37 min., 22 sec. 

with E-2C leading. 



4 

Determine whether a 
time delta between Link- 
16 participants impacts 
either the weapon system 
or SlAP performance. 

Insert a time delta 

between the Utility 
player and E-2C mission 
computers. 

Recommended time delta 
is 1 hr., 37 min., 22 sec. 

with E-2C lagging. 
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5 thru 
TBD 

Parametric analysis of 
Azimuth/Yaw bias. 

Insert a (+/-) AZ bias 

keeping all other biases 
constant. Adjust {+/-) AZ 
bias, as required, to 
determine the level of 
bias that impacts E-2C 
mission and/or the SLAP 
attributes. Note; It is 
estimated that it will 
require a total of 5 to 7 
runs to determine the 
bias "Knee in the curve”. 



6 thru 
TBD 

Parametric analysis of 

Geodetic Registration 
impact upon Sensor 
Registration North biases 

OSRP latitude position. 

Insert a geodetic latitude 
error for each 
subsequent run until the 
maximum uncorrectable 
bias offset has been 
determined. Example: 
Increase the position 
bias offset from truth in 
a logical methodology as; 
100 m, 300 m, 1000 m, 
10000 m, and 30000 
meters. 



7 thru 
TBD 

Parametric analysis of 

Geodetic Registration 
impact upon Sensor 
Registration East biases. 
Note: This series of runs 
might not be reauired if 
the SLAP attributes 
eouallv imnacted bv error 
regardless of the x and v 
bias sign (+/-). 

OSRP longitude 
position. Insert a 
geodetic longitude error 
for each subsequent run 
until the maximum 
uncorrectable bias offset 
has been determined. 
Example: Increase the 
position bias offset from 
truth in a logical 
methodology as; 100 m, 
300 m, 1000 m, 10000 
m. and 30000 meters. 



8 thru 
TBD 

Parametric analysis of 
Range bias. 

Insert a (+/-) Range 
bias keeping all other 
biases constant. Note: It 
is estimated that it will 
require a total of 5 to 7 
runs to determine the 
bias "Knee in the curv-e". 



9 thru 
TBD 

Parametric analysis of 
Range Rate bias. 

Insert a (+/-) Range 

Rate bias keeping all 
other biases constant. 

Note: It is estimated that 
it will require a total of 5 
to 7 runs to determine 
the bias "Knee in the 
curve”. 
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Table H-2 Alternative Run Matrix 


Run 

Test Objective 

Parameter Being 
Set 

Value 

Responsible 

Party 

3A 

Determine whether a 
time delta between the 
E-2C network radio 
(JTIDS terminal) and the 
mission computer 
impacts either the 
weapon system or SlAP 
performance. 

Insert a time delta 

between the E-2C 
network radio (JTIDS 
terminal) and the 
mission computer. The 
JTIDS terminal shall be 
synched to GPS and the 
mission computer shall 
use a false local time 
with a delta of 1 hr., 37 
min., 22 sec. with 

JTIDS terminal (GPS) 
leading. 



3B 

Determine whether a 
time delta between the 
E-2C network radio 
(JTIDS terminal) and the 
mission compute 
impacts h 

weapon s 

performar ' t i 

Insert a time delta 

between the E-2r 
network r fi 

^ rr U % i 

^ "K ' ^ '%■ 

1 i 1 1 * \ 

'|ii ^^2 

'IL.J terminal (GPS) 
lagging. 

% 

3 'M0f^ 


3C 

Time Synch, ai a 10 sec. 
time delta impacts E-2C 
operations perform a 
dynamic parametric run 
varying the time delta in 
the following increments 
every 5 minutes. 

Increase the time delta 
every 5 min. between the 
E-2C network (JTIDS) 

NTR and the mission 
computer clock in the 
following increments; 1 
ms, 3 ms, 10 ms, 100 
ms, and 300 ms. 



3D 

Time Synch. E-2C is the 
master clock reference for 
the network radio using 
GPS time 

JTIDS Terminal: NTR 

GPS time (Enabled); 

Host System: 

GPS time (Enabled) 



3E 

Time Synch. E-2C 
network radio is slaved to 
a NTR. The remote NTR is 
slaved to GPS as is the E- 
2C host system 

JTIDS Terminal: Slaved 

GPS time (Remote); 

Host System: 

GPS time (Enabled) 
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3F 

Time Synch. E 2C 

network radio is slaved to 
a NTR. The NTR Is using 
a relative time reference 
while the E-2C host 
system uses GPS time 

JTIDS Terminal: Slaved 

Relative time (Remote), 

Host System: 

GPS time (Enabled) 



3G 

Time Synch. E-2C 
network radio is slaved to 
a NTR. The NTR is using 
a relative time reference 
while the E-2C host 
system uses its own 
relative time reference. 

JTIDS Terminal: Slaved 

Relative time (Remote), 
Host System: 

Relative time (Enabled) 



6A 

Nav. Reg. Operate E-2C 
use GPS as the only 
Navigation reference. 

Use GPS reference 



6B 

Nav. Reg. Operate E-2C 
use INS as the only 
Navigation reference. 

Use INS reference 



6C 

Nav. Reg. Operate E'2C 

using a blended 
"combined" GPS & INS 

Navigation reference. 

Combined reference 
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Test Procedures 

Provide the t est procedures in Table H-3 and indicat e whether these are the final procedure or whether 
they will be updated following the Integration process. 

Note: Having on-site procedures and log are useful to clarify the minimum data collection goals for critical 
experiments. 


Table H-3. Test Procedures 


Pre-Test Setup 


Seq. 

# 

Sequence Function 

Step 

Action 

Acceptance Criteria 

Pass 

Fail 

1 

Setup MC^ to ^rSlU 
configuration 

1 

a 

Configure the patch panel and cabling to 
connect the MC and TSIU/E:-2CTD 



~2 

Boot host 

2 

a 

I'SIU bootable image host system. 




computers 


b 

RiSS E-2CTD (3 hosts: E-2CTD, Radar 

IFF) , % 1 






c 

Tactical System Intc , ti: 

. 





d 

HLAuuLfy;;.^ \ .l \, \ 1 \ \ f 






C' 

/. 'i 'V'" % 1 1 • Is 1 \ 

1 Vr '■ \ ^ V \ 

^ 1 », v» 1 \ 1 ^ « \ 

% 1 !*> <!y' "f * S 1 ! 

i 

1 





T" 



3 

Power up MC 

3 

a 




bench 


b 

' '5t i> 4*' 






c 

‘ ' 






d 

1 L e bench 






e 

Select and boot the ACIS 






f 

Power the MFCDIJ 



4 

vStart TSIU 

4 

a 

Boot the TSIU and verify it is using the 
following configuration: 
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Pre-Test Setup 

Scq. 

# 

Sequence I- unction 

S1 

:ep 

j Action 

Acceptance Criteria 

Pass 

P'ail 





- Simulation Mode 

- Nav Upgrade 

- RISS 



5 

Ix)ad RISS E-2CTD 

b 

a 

Start the RISS P2-2CTD and verify the 
following configuration: 

- RISS enabled 

- Radar in operate mode 

- Radar channel 5 

- Transmitter power = 100% 

- PD = 100% 

- Antenna speed = 6 RPM 

- TACCAR enabled 

- DPCA enabled 

- ECCM enabled . ^ f, % 

- Gaming area,,-: S %, 

- EvfU /fy' /,v s 4 % \ 1 1 

\ 1 Itl h L 1 1 1. 1 



6 

Start Link 16 GTE 

6 

— - 

\ L M..,, i. \, ft 1 



'•i 


u 

c 

W h V A li ^ j 



. 1 1, 1 



7 

Load HLA I/O GW 

7 

a 

1 ' 1/ . ’GaLway and verify it is 

configuration: 
?'tlDEP/SlAP FOM 

- Connected to RISS E-2CTD 

- Using valid UTC time 



o 

IXiHa 2D Viewer 



Load the 2D Viewer and select the 
JDEP/SIAP FOM. 



y 

initialize HLA OX 



Load the hlaResults logging application 
with the JDEP/SIAP FOM 

-,_ 
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Pre-Test Setup 


Seq. 

# 

Sequence FuncUon 

Step 

Action 

Acceptance Oiteria 

Pass 

Fall 

10 

Establish Voice 

1 


Establish communications with TCC via 




Com ms 

1 


selected comm, systems (ASTI or 







SphereCon) 
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Pre-Brief and Network Verification 

Secj, 

# 

Sequence Function 

Step 

Action 

Acceptance Criteria 

Pass 

Fail 

1 

RID File Check 

1 

a 

TCC reviews RID file for day's runs 

Each federate ensures their RJD file is 
correct for day’s testing. 


2 

IPMC Test 

2 

a 

TCC directs each federate to run IPMC 
Tester as sender, while all others receive 

Each receiving federate receives all 
packets sent by sending federate. 


3 

Time Synch Check 

3 

a 

TCC checks time synch 

Time marks match TCCs (IRIG) 


4 

Confirm Site 
Readiness 

4 

a 

TCC asks if any sites not r ' fr 'i 

> . \ \ S %: 1 

Each federate verifies capability changes 


5 

vScenario Readiness 
Verification 

b 

a 

^ " 1 1 t \ ‘1 scenario readiness status, 

-A ' . Vh? {''t 1: 1 4 1 1 1 i 


6 

Confirm TCC Setup 

6 

a 

t « k k- t- t' f \ 1 1 

^ ' T‘ % '3 \ ■ 1 11 1 

i '.t> vs M4. 'mW-- 


;p version is noted 


7 

Cfonfirm Site 
Configurations 

7 

a 

; ify all federate configurations from 
♦i™ lists 


8 

L I 6 NTR 

Established 

8 

a 


' % % 0^ ^ ■S**ei.vl/6rk as NTR 

AEGIS verifies initiation of Link- 16 as 

NTR 


b 

E-2C confirm in Link 16 in data silence 
** IBAR verity in Link-16 in Data silence 

** ACETEF verify in Link-16 in Da ta 

Silence 

--- 

E-2C confirms they are in Data Silen<:;e 
IRA13 confirms they are in Data Silraice 
ACEITiF confirms they are in Data 

Silence 

** only one ver. 8.4 GTE can be running 
at a time 



UNCLASSIFIED 


Page H-8 

7.2.7.2_TRR(04-016)_ L0Z_JSSEO_041210 




UNCLASSIFIED 


Federation Initialization 

Tliis section is an nnknowa'i at this point ... this needs to be updated with the actual steps to get the lederation initialized until we 
know what federation management tools and processes will be used can t l:>egin to till this in ~ it is left in our original ionnat -- Jirst 


to Indicate wtiat previously was here. 


Seq. 

# 

Sequence Function 

Step 

Acdion 

Acceptance Criteria 

Pass 

Fail 

1 

.Start Link' 16 

Loggers 

1 

a 

Verity Link 16 DX ready. 

i'e{x,)rdlng trial # „„„., 

with a];)proj)riate track quality 

Jiife - I, 

.... \ \\ \\ 

crd.... . a ..4 .. .. \L i i 

India 

Victor 

F14 

’■12 

JXllS 

■ iggested filename: flmmdd tt| 


2 

PPLl ISntry 

2 

a. 

/ 'A"% 't: \ verifies thciy are out of data silence 


L 


1 

1 %erifies they have entered Link-16 


c 

h h 1 ^ 

SMM'verific's Link-16 is active 


d 

Vu . \ f. ' 

Victor verifies Link-16 is active 


e 

All % ^ ..i^ci}:Mfts verily they see all 

{,)thc,t||§af«iclpants in Link- 1 6 with correct 
PPLl' 

India 

Victor 

FI 4 

E2 

AEGIS 


*> 

. Join/ H(;alth Check 

3 

a 

TCC begins RTI exec. 

RTI console list verifies join and USD 






Joins HLAResults 

validates Health check. 






Joins USD / 'rCF 

Trial # 





L 

TCC directs C federate to Join 






c 

TCC directs 2"^’ federate 1:o join 






cl 

TCC directs IF'! fe^derate to join 






e 

TCC directs 4'** federate t:o join 






f 

TCC directs Federate 1:o join 
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Fede 

Tins s 
know 
to ind 

ration Initialization 

eclion is an unknown at. tills point ... tiiis needs to be updated with the aetua! steps to get thf* federation ini(:ializ(;;d - until we 
what, iedei at.ion .tnanageiuent tools and proc';esses will lie used can’t bfigin t.<i Jill this in - it l.s lefi: in our original fonnat ••• just 
cate what f)rev1oi,islv was here. 

Seq. 

# 

Sequence Function 

Step 

Action 

Acceptance Criteria 

Pass 

Fail 




g 

TCC directs 6**' federate to join 



h 

TCC directs 7'*' federate to join 



I 

TCC directs S'*' federate to join 



J 

TCC directs 9"’ federate to join 



.) 

TCC directs Viewers to join 



4 

Start Sequence 

4 

a 

b 

c 


'|cia and IBAR verily that MNS- 1 link 
i. 1 terminals <:i.re fnnctional 

1 1, 

t- 1 . 


^ ^and Vkdnr verily that aircraft have 
% i)vt;r and are flying 

% .<K 

h (('derate simulation l.>eglns running 


5 

IFF Interrogation 

5 

a 

■0i < 

Message is i)ublished by E-2C, as 
verified ifom data log hies. 


6 

Entity State 

t:> 

a 

1’CC verifie.s entities are displayed at 
correct scenario locations throughont: 
scenario 

Verifies using 2D & 3D vit'wers (& 

RCCSII display] that entities are in 
orbits and wingmen are following leads. 
Reports anornalitis when noted 



UNCLASSIFIED 
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Federation Initialization 

'Dus section is an unknown at: this point: ... this needs to be npciated with the actual steps to get tlit? federation initialized.until we 

know what federation roanagernent: t ools and processes will be used can’t: l:>egin to till this in - it is left in our original iormat - just 
to indicat:e wiiat j::ireviouslv was here. 


Seq. 

# 


vSequence Function 


Action 


Acceptance Criteria 


Pass 

Fail 


IFF'' Ftesponse 
Interaction 


(.Join Viewers as E2 
Si AECIS verily IFF 
interrogate) 


152 verify all traeks are being displayed on 
the E-2C tactical system and f:oiTect 
positions and Ids are being reported 


E-2C verifies correct 
aircraft cockpit displ 
designations 

I II 

62 230^ 

62 240;) 

I' I 


IFF' codes aiifl 
ays have correct IFF' 



Yes 


|i 

CidAirLR 

AEGIS 


Civ Air 


III 

IV 

C/Alt 

3400 

End 

Yes 

3400 

F'nd 

Yes 

600 D 

No Res 

^pYes 

3207 

f>0/A 1 

F'nd 

f] 

Yes 

1 

3401 

t- no 

I'nd 

I Cb 

Yes 

3402 

End 

Yes 

6002i 

No RespYes 

5210 

End 

Yes 

No 

F'nd 

No 

i/a 

6003 > 

No Resp 


AEGIS verify all tracks are being displayed 
on your tactical system and correct 
positions and Ids are lielng reported 


AEGIS verifies all IF'F' codes are 
correlated and tieing reported correctly 


Plai:form 

Configuratic:)!! 


8 


TCP' uses 3D display l:o verify correct 
visual representation and orientation of 
each entity 


TCF 3 D viewer displays c;orrec1 platform 
configuration, as verified liy test 
controller. 
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E-2C 

Tactical Operate 

r Procedures 

Seq, 

# 

Sequence Function 

Step 

Action 

Acceptance Criteria 

Pass 

Fail 





THE FOLLOWING PROCEDURE WILL BE 
REPEATED FOR EACH RUN OF THE 
DATA REGISTRATION TEST MATRIX. 











Pre-test Set-up 



Radar; Ch. 5, Medium PRF, 6 rpm 

Verify on ACIS. 






IFF: Mode I, II. Ill, C and iV 

Verify on ACIS. 






Videos: As desired. Recommend 
monitoring RTSV and PSV. Save setup if 
desired. 

Verify on ACIS. 

File ame 






Select'I’entative Tracks ior'•’is- ^ * ACIS, 

Track Display S^Tq. < ^ ." ' | 1 |. 






Prehoo. y '^u 

display V 1 1 1 '1 1 ^ 

- 4 ■ 1 w 4 V 

tt ’It fx f> % ''s '5 

v^'-f ■i"-^ "M m 

1 1 1 1 . 

1 * 1 1, 

1 1 ^ 

5. A 

6. CAN 

7. Filter No 

8. Miss Counter 

9. Brg/Rng 





Geo Points: Enter geographical points iaw 
CRS Data Registration Excursion.Save 
load. 

Verify on ACIS. 

File name 






Expected Pi: Enter expected Pis on all 

Blue aircraft iaw CRS order of battle. 

Blue Force Mode 11/Platform lype: 

Obiect 1: II Class 

Object 2: II Class 

Etc. 
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E-2C Tactical Operator Procedures 

Seq. 

# 

Sequence Function 

Step 

Action 

Acceptance Criteria 

Pass 

Fail 





Link 16: Enter ownship Jl'N, JTN limits, 
JTIDS Init specified load, as assigned. 
Auto-assoc tracks enabled. Auto-report 
local trks enabled. 

Verify on ACIS. 

E-2C TN 

E-2C TN Limits 

An to-Assoc 

Auto-Report 

JTIDS Load 






DX: Select DX points, 

1 u \' 1 

\ 1 ^ 1 1 1 1 w 

^ 1 1 1 * 

1 1 

1 m 

DX Points: 

TrkFile 

RptFiF 

% % 

i::-. % 1 
mm\ \ 
ii>l 1 1 

«■; III 

Ifl ■ 1 * , 

4. < 1 % 

li.:'-.- 1 'II 1 

Out 

LI 6 Rev 

LI6 Xmit 

LI6 Dbase 

L16 Prmtr 






start DX. Enter DX file name as 

SIAPDR X, where the X represents the 
test matrix run number. Select REC to 
start DX. 

File on hard drive. 

File name 
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E-2C 

Tactical Operator Procedures 

Seq. 

# 

vSequence Function 

Step 

Action 

Acceptance Criteria 

Pass 

Fail 


Scenario Start 



Perform Link-16 net entry. Verify receipt 
and transmit of link tracks. Select RADIO 
SILENCE and perform TRACK FILE 

CLEAR. 

Verifies system operation. Clears track 
files so test will start with new data. 






Come out of radio silence, by selecting 
NORMAL mode. 

Verify on ACLS. 






Monitor Data Registration window, note 
Ownshlp Correction pad values. I.cave 

Data Registration Window up 
continuously, if feasible. 

Verily on ACIS. Periodically record pad 
values. 

Pads; N/S E/W 






Monitor and verily local and/or remote 
track on all CRS tracks. Monitor detection 
and acquisition of all Blue/Red tracks law 
CRS. Manually ID Blue Force objects as 
tracks are established if expected PI 
function fails. Red Force trat;ks will be ID 
law ?????? 

Verify on ACIS. Compare to CRS object 
list. 

(Need object list} 

, % t- 
111 






Verify auto-report ing of local tp- ks - 

Link-16. Manually 1 ' 'n 

auto-report;- V U ,, ? 





Verify auto-c- % 1? \ ' fli ^ | ^ 

local and ren ; t\st Ilf % 'S li 

associate and 

1 1 1-- 

tt % % 

■ i || \ || 





Monitor Link c \ 1 | 1o lb J 

track correlatic i 

unexpected eve. 





Monitor dual tracl:'' situations and note 

JTNs and ACNs, as appropriate. 











Scenario FINEX 



Upon FINEX secure DX. Reconfigure for 
next run. 

-—-- 
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APPENDIX I: DETAILED ACTION PLAN 

The action plan should provide a detailed schedule for administrative use. 

Table I-1. Detailed Action Plan 


Tasking 


July 


August 


September 


October 


November 


I. Define Federation Objectives 

1. Understand analysis needs for data registration critical experiments and impact on construct 
of the federation (SIAP-Votruba. Youmans, JITC-V'FC, JITC-FHU) (Youmans is responsible for 
overarching DMAP) (8/9) 

2. Define attributes and MOPs for data registration and the data needed from the federation to 
support analysis (SIAP-Votn.iba, Youmans,JITC-KHU, JITC-VTC, E-2C, 8/30) 


X 


3. Understand Analysis Infrastructure (metrics, tools,.,) (SIAP- Corona, Votnjba; JITC-FHU; JITC-VIXI , 8/16) 


II. Conceptual Analysis (scenario; conceptual design, federation requlrementsi) 


1, Document conceptisal analysis (identify in pictures or in,;^v.' ^ It 

fidelity. At tiie same time, begin doc-unientatiopj''^ 

"''■‘'Ji'-'"”-''. 

2. Coordinate with the CRS Team to 

( 8 / 22 ) ..^.,.. 

a. CRS Team to propose reference.. .'’'2 


.... 

-. 0 . 


' > V: r r 


ts 


I 


1 

ll 




I 


"',d 



liid at what 
^ITC-FHU; 9/3) 

X 


I I I 

H ii 


b. Review of scenario at SPA (CRS;: . . 

3. Detennine and document federation i 
(dITC-VTC:9/5} I, 1 

, % % 

a. Determine system requirement:/rep. m | 

% % 

b. What is required of a scenario playb, || 

c. What is needed for Data Collection an^^Tederatlon Managenient? (JITC-VTC SIAP-Corona) 9/5) 




II) { 


% 

% I I 

« I 'I 

management) 


d. Equipment and computer program requirements to E-2C (9/6) 


X 


X 

X 

X 


III. Design E-2C Pilot Federation (select federates, federation design, FOM, federation agreements, federate design, implement federation) 
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1. Determine metlioci for scenario generation and playback, {JITC-VTC, jnX>FHU} (9/5) 

2. Designate Federates and representational responsibilities (8/23) 

a. System representation (dlTOFHU, E-2C) 

C.onduct pre-verification of system rejiresentation 

b. Generic system representation (TBD) (JITC-FHU) 

d, Data Collection (SIAP-SE, E-2C, Corona] 

e. Federation management (JITC-FHU! 

3. Develop test plan (SlAf>-SE, E-2C, JITC-FHU) 

a. Test plan outline published (SIAP-SE: 8/28) 

b. Test Procedures {E-2C; 9/3) 

c. Test Procedures (E-2C; 9/9) 

d. Signed Test Plan (SIAP-SE: 10/16) 

e. Pre'fest Readiness Review (SIAP-SE: 10/22) 


f- Test Readiness Review (SIA1">-SE: 10/29) 
g. TPWG#3 (SIAP-SE; 7 } 

4. Develop FOM, using conceptual analysis, DMAP, design rules, and pcrfo-->nan'-<i 

considerations. Map against the SE REF FOM (JVB/JSB merged V' c 
,002:9/5; final 9/17) ^ 

5, Document how federates will meet feder ;.", re - - 
bias, time synchronization.,,) (E-2C,iO/9) 


■ 






6. Describe the Ongoing W&A approach (E- 

7. Determine security needs (need network, e 

rv. Develop E-2C Pilot Pederation 

1. Upda te tederates to support federation need; 



iei. 


lit:a 


X 




Ic 




X 



^ (e.g., sensor ^ 




Y. 
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1 , Develop Federation ImpiementaOon Plan j 

a. Determine Federation Agreements. (Specify management scheme, version c 
management schema, deliver mechanisms, use of DDM) (JITC-VTC, 9/17) 

b. Integration/Test Schedule (JiTC-VTC; 9/27] 

c. FEPWA (JITC-V'rCi; 9/27) 

2, Generate te.st report (SIAP-Youmans; JTIC-FHU) (9/22) 

3, Integration and Testing (10/15-11/!) 

1 

f RTl, data marsh 

ailing, time 

X 

X 

X 

X 

X 

X— 

X 

VI.Execute Federation 

1, Run Event; collect data (E-2C ,11/4-1.5) 

2. Condi.ict W&A {10/1,5-11/1) 




X- 

X-X 

X 

VII. Analyze Data (SIAP SE. 11/4-12/31) 

Quick look report and review session (All) 





X- 
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APPENDIX J: FEDERATION DEVELOPMENT {M&S Venues) 

Appendix J provides further description of the test federation than 
what was discussed in Section 2. 

For example, “Simulation, ground truth, and control data will be 
communicated across RTl NG Version 1.6 and Link 16 traffic will be 
exchanged via simulated network using SPAWAR gateway terminal 
emulators (GTEs). Figure J-1 represents the resulting architecture." 



Link 16 tracks and PPLIs 


MLST-3 

(Utility 


CRSD 

rScvnario Driven 

■Kvd Pmitional il4<a nnd 
iiiili designation 
■Slue Positional data and 
»ide dtiifgnaliufi 
■I* 1( piivi(.ur..il diM 
•Provide If h resiwmsc to 
Intc rrog-klion Pc quest 




• . crrogation 

iCaii/aiion, nimitoring.. 


fO version 1.6 ReL 
ssponse messales. ai 


'fo.st 

Control 

Federate 




|?b \ ewei 


□ Support tools Link 16 EmulationI—J Simulation HI HWIL 

Figure J-1. SLAP HWIL JDEP pilot federation 

Provide further details regarding system representation, interface 
and information processing. Provide an overview followed by a specific 
breakout of each system. 

For example: 

Federation Tools: The ESTEL tool suite provides data collection, control 
and views of the federation. For this event, the Test Control Federate 
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and the viewers will be run from a single workstation along with 
hlaControl and hlaResults." 

For example: 

Test Control Federate: The Test Control Federate (TCF) provides the 
ability to create/destroy the federation and to issue Start and Stop 
scenario commands. It has the ability to support a display of each 
federate's status." 

For example: 

2D Viewer Federate: The 2D viewer provides a plain view display of 
federation and all active units. It is an adaptation of a 2D viewer 
originally developed by Meta VR. It uses ADRG (Arc-second Digitized 
Raster Graphics) for map display." 

Figure J-2 gives a notional example of the federation object model object 
classes. 
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SimulationSer 


Miiiisijier 


Ke(|uest 


Figure J-S shows a notional set of Interactions for a typical federation. 


IM Iiiti rrni!:iii... 

►_I 


Figure J-3. FOM interaction classes 


Figure J-4 shows the federation key attributes for a notional FOM. 
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I’liitiiii 111 


IndividuaCom, 


GroundPlatform 


Aircraft 


Munition 


Hi^objectKoot 


qiupiiii' 


Uadur Lniitler 


Uadai^icibor 


I ederation 


M.iiiaai’i 


Figure J-4. FOM key attributes 


Tables J-5 and J-6 show the publish/subscribe activity of the 
federation components for the object classes and their interactions. 
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Table J-1. Publish/Subscribe Activity of Object Classes 


FEDERATE 

rj 

v> 

VI 


G. 

as! 

a 

o 

o 

a 

5 

■. f 

% 

^ i 

\ % 

• 5 

%. 

A 

TIAC 

s 

7^ 

I 

OBJECT CLASS 

1 . - - ^ 

lEauiDment.Sen ^ t 



B 

Equipment.Navii 

Platform.Aircraft 

Platform.Suiface\ 

-• ,... 

% 

1 

% 

% 

J 

1 

1 

1 

B 

B 

Plaitform.GroundPi 




iCCMiSlWWS 


■ 

■ 


Platform.UGS (PS) 


m 



B 

iB 

B 

Bi 

B 

Platform.Munttion (1. 

P 

m 

B 


»i 

■ 

m 

m 

B 

Platforin.lncjHvidualCombatarrt (PS) 


m 






— 

BI 

FederateStatus 

m 

m 

m 


s 



-p- 


‘ = OPTIONAL 










P = Publish 









1 S = Subscribe 





] 

i 
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Table J-2. Publish/Subscribe Activity of Interaction Classes 


FEDERATE 


i 

i 

i m 
' ro 

n 

utility Player 

i 

H 

1 

n 

^ s 

Q 

s. 

o 

i 

TIAC 

3* 

» 

7S 

i 

ifi 

IHTERACTION CLASS 

o 

so 

w 

1 ^ 

hlaControl 

s. 

! 3 

i "n 

: E. 

1 

n 

IPC 

1 

— 


'1 

■.• 

• 

■ % 

"i 1 
; 

L 




StartTestExecution {IR> 


r 


1 

1 

s 

S 

s 

StopTestE*eciiti''n (' 

It r 

r' “ 

'■I— X— 
1) : 


s 

s 

s 

StartiniineL h 



s 

StopfniineLi 

* 


s 

initiaiizaitjonL 


s 

s 

InitializationDi 


s 

s 

StartScenarioL 


. i 



P 

~Ws 

i: 

■ 

s 

Re^sterFedert 

1 

p 


p 

S 




s 

AcknowledgeU ra. 

S 

S 

m 


P 




5 

RegisterHeatthStatusftem 

p 

p 

p. 

p 

S 




s 

SimiilationSeruice 










Propagated (and subclasses) 










Perception (and subclasses) 










Detection (and subclasses) 










IFF 










IFFJnterrogation (IR) 

s 

p 

s 





s 

s 

IFF_Response (IR) 

p 

s 

s 





s 

s 

*" OPTIOHAL 

i 



j i 

i 



P = Publish 




1 j i 


S = Subscribe 

.."i-:• 



....„] 
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APPENDIX K: DATA ANALYSIS PLAN 

The data analysis plan includes data management, extraction 
diagrams, extraction point tables, data formats, archiving, and any 
additional information on the measures of effectiveness or measures of 
performance that was not already addressed in the critical experiment 
discussion in Section 2.2. 

ORGANIZATION AND MANAGEMENT 

This section should identify the organization of the analysis effort. 
Depending on the venue (live vs modeling and simulation), provide a 
description of the appropriate roles of key functions. These may include 
Data Analysis Manager, Data Collection Coordinator/Manager, Site Data 
Coordinator, Site Leads, Test Director/Site Test Directors, and Event 
lead analyst. Assign names and responsibilities for each function. 

DATA RECORDING AND COLLECTION 

Provide a brief description of the data collection and management 
process. 

Success Criteria 

Discuss success criteria and who determines if the criteria are met 
based on what data provided. 

Automated Data Management 

Describe any tools for automated data collection. Provide a table, 
if necessary to describe tools used by each system Involved in the test. 


Recording Media 

Describe the recording media used by each system. Use a table if 
necessary. 

Data Extraction 

For each system and any other elements participating in the test 
(e.g,, infrastructure, truth module), provide a data extraction diagram for 
collecting the data required for the test. See Figure K-1 for an example. 
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Figure K-1. Data extraction diagram. 


Link 16 Tx/Rs, & TIM/TOM Data 


Link 11 Tx/Rx Data 


Purpose of data extraction; 
An = Attribute Calculation 
RC = Root-cause Analysis 


DISPLAY 


(1) For MCU, recording of IT % |il ^ /Vters UniL- 11 VQ Buffers recorded on computer 

(2) For Group [I, higher fidelit> d f vpe I or 2 recorder is installed. One type of recorder offers the greatest detail of 

recording bin: with a significant requirement to physically connect to different sysiems in the aircraft 

(3) Start and stop of data recording controlled via operator/contpuier for both. 


JTIDS 

TERMINAI 


If desired, provide a data extraction table listing the location, 
name, and function of the extraction points. 


Test Observation Reports 


All test observations such as anomalies, events of interest, or any 
problems should be captures in a test observation report (TOR). A 
sample TOR form is provided in Appendix L. Describe the TOR 
adjudication process; that is, how TORs are assigned for further analysis 
or whether the TOR should be entered into the Lessons Learned 
Knowledge Base. 


Manual Data Collection 


In this section, describe the minimal requirements for recording 
manual data, which includes completing the chronological log (if 
applicable), annotating test run summaries and procedures, generating 
TORs, and labeling automatic data extraction media. 
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Recorded Data Labeling 

Indicate how the data recorded will be labeled. Include labeling 
plans for tapes, CDs, optical disks, and any other media storing relevant 

test data. 

Data Transfer 

Describe any data transfers that will take place during or after the 
test. Include information on encryption or electronic means required. 

File Naming Conventions 

If called for, indicate any file naming convention that will be used 
for the test to facilitate locating data among test participants. 

DATA ANALYSIS PROCESS 

This section describes the data analysis process, including 
calculation of the SIAP attributes and critical experiment analysis. 


SLAP Attributes 

The definitions of the SIAP attributes were provided in Appendix B. 
In this section, describe how the SIAP attributes will be computed, 
including use of the Performance Evaluation Tool (PET) and the 
Automated Reconstruction and Correlation Tool for Interoperability 
Characterization (ARCTIC). 

Measures of Performance 

In this section, provide a detailed description of the measures of 
performance (MOPs) defined in Section 2.2. When available, provide 
mathematical derivations and the steps to compute the MOPs. 

Provide notional charts as fitting. For example, Figure K-2 denotes the 
number of correlations achieved as a function of azimuth bias applied. 
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T 



Azimuth bias (milliradians) 

Figure K-2. Notional number of correlations vs azimuth bias 


DATA COLLECTION AND ANALYSIS PLOW 

Provide a diagram showing the data collection and analysis flow for 
the test. See Figure K-3 for an example. 
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Figure K-3 Notional data collection and analysis flow 
diagram 

DATA REDUCTION/ANALYSIS TOOLS 

For each system participating in the test, provide the data 
reduction tools to be used and a brief description for each. 

DATA ARCHIVING 

Identify which organization will retain the data and the timeframe 
for doing so. Identify a point of contact to whom inquires should be 
directed, and provide voice and e-mail contact information. 
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ANNEX K-A Chronological Log 

This annex contains an example of a form that should be used to 
record the chronological list of events as they occur during testing. Items 
to be recorded include: test observations, events of interest, changes in 
configuration or equipment status, and any other information that would 
assist in the analysis: 


Date: (Calendar and Julian] 

Test: 

Site 

Site Test Director: 

Data recorder 

Tactical Operator 

Tactical Operator 

Time lU 

Comments 
































ANNEX K-B PET Input Format 

Data Required for SIAP Attribute Calculations 

In this annex, provide details regarding how SIAP attributes will be 
calculated. Provide version numbers of computer programs used (Table 
K-B-1) and formats (Table K-B-2). For example. 'There is a discussion of 
the ESTEL data reduction tools in Appendix D. Prior to the test, Corona 
will provide updated PET and ARCTIC tools and training. Table K-B-1 
lists the program version of packages that will be used for analysis." 
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Table K-B-1. Computer Program Requirements 


Computer Program 

PTinction 

J»&.. 1 


Provider 

E-2C tool suite 

Data ro'li 

> ; ' ’1 t 

% 

ESTEL 

PET 

^ 4 V : 'b 


W- 






Corona 

ARCTIC 

H 








Corona/CNA 


Table K-B-2. PET Input Table, WAM Format 


VARIABLE 

DEFINITION 

NOTES 

DYNAMIC 

RANGE 

(TBD) 

SYS 

System Variant 

30 = E'2C 



CTSL 

This is tile number used to 
identify tracks in r’ co -c - 
system'^.A i . yc ^ % %, 

‘ y j '' 1 ^ ir 'Sy ^ 

i "ri. 1 ¥ 1 



LTN 

^ %■ %: 


XT 

t 1 1 ’ 

"r f" ' 

■00 


DVALTIME 


% U 

OSLAT 

% ' % % ^ ^ ^ 
\ '§^1 % - foT* 

% % ***' 

1. r t: - DD.ddddd 

Positional 

Information 


OSLONG 

own Ship Longitude in degrees 
Positive for East - Negative for 
West 

Format: +/- DDD.ddddd 



OSALT 

Own Ship Altitude in feet 



OSHDG 

Own Ship Heading in degrees 



OSSPD 

Own Ship Speed in knots 



X 

X Y Z Distance from own ship 
reference center in data miles 

Track 
Positional 
Information 
Method #1 


Y 




Z 
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XVEL 

X Y Z Velocity in data miles per 
second 

Track 
Kinematics 
Method # 1 


YVEL 





ZVEL 





LAT 

Latitude in degrees 

Positive for North - Negative for 
South 

Format: +/- DD.ddddd 

Track 
Positional 
Information 
Method #2 


LONG 

Longitude in degrees 
Positive for East - N^gat 
West .. 

r na 'O '1 


ill 1 


AI.T 


i \ I i \ 


CRS 



1 1 1 1 
^ 


SPD 

. 





CLM A 

t ; ’|e "pt ^ 

'-'I 

1 


CAT 

'1 ’|J. ec 

|i ,j5»s^r 

sl'^-^urface 

3 = Subsurface 

4 = Land 

5 = Space 




ID 

Identification 

0 = Pending 

1 = Unknown 

2 =: Assumed Friend 

3 = Friend 

4 = Neutral 

5 = Suspect 

6 = Hostile 

7 = Undefined 



LTQ 

Local Track Quality 

0-15 = 0-15 

Track 

Quality 


RTQ 

Remote Track Quality 

0-15 = 0-15 



MUTRK 

Mutual Track Indicator 

0 = Not Mutual 

1 = Mutual 



LR 

Imcal or Remote 

0 = Ivocal 

1 = Remote 



RIJ 

Reporting Unit 

The LTN of the unit reporting 
this track 



Ml 

Mode I 

Prack IFF 
nformation 
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M2 

Mode II 



M3 

Mode III 



M4 

Mode IV 

0 = Not Interrogated/No 
statement 

1 = Interrogated, No response 

2 = Interrogated, Invalid 
response 

3 = Interrogated, Valid 
response 



D1 

DI Code 



SIZE 

Size/Strength 

0-15 = 0-15 



TRKST 

ENG 

OSENG 

Track Statn®,- \ 

0 = 

V 'pi _ A % - 

^ ^ ^ 

^ ^ |sl ^ ^ 

5 =J1'arget Destroyed 

6 = Partially Effective 

7 = Not Effective 

8 = Engagement Broken 

9 = Heads up 

10 = Engagement Interrupted 

11 = 

Investigating/ Interrogating 

12 = Shadowing 

13 = Intervening 

14 = Covering 

15 = BDA unknown 

\ 1 1 

■: 1 % 

^ ^ 

S-: 

% 
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RENG 

Remote engagement status 

0 = No statement 


1 


1 = Recommend reattack 

2 = Weapon assigned 

3 = Tracking 

4 = Firing 

5 = Target Destroyed 

6 = Partially Effective 

7 = Not Effective 

8 = Engagement Broken 

9 = Heads up 

10 = Engagement Interrupted 

11 = 




Investigati ng/1 nterrogating 

12 = Shadowing 

13 = Intervening 

14 = Covering 

15 = BDA unknown 



TRKSRC 

Track Source 

Track 



0 = Source N/A 

source 



1 - Link 4A 

2 = Link 11 

3 = Link 16 

4 = Link 16 Down Link 

5 = IFF 

Information 



6 = Manual ■ % 1 

;. ■ % 1 



7 = SPY , ~ \ t % 1 



,8 ^ \ \ \ 1 
~ ^ 1 4 1 \ 1 



^ 

111 1 



: % X % 

? ' , k. ^ ^ 

vj 1 \ 1 



% ‘ % % 'R « 

% 

i 



1-. \ % \ 

% 


1 '■ ' % % % ® ' 

i 1 
i. 



% ^ ^ '% 

1 Ie- 








20 = SLQ 

21 =SQQ 

22 - wSQR 

23 = SQS 

24 = TAG 



MISRC 

Mode I Source 

0 = Information Unavailable 

1 = UPX 29 

2 = Link 

3 = Manual 

4 - CEC 

5 = ecu 

6 = Link 16 PPLl 
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M2SRC 

Mode 11 Source 

0 = Information Unavailable 

1 = UPX 29 

2 = Link 

3 = Manual 

4 = CEC 

5 = ecu 

6 = Link 16 PPL! 



M3SRC 

Mode III Source 

0 = Information Unavailable 

1 = UPX 29 

2 = Link 

3 = Manual 

4 = CEC 

5 = ecu 

6 = Link 16 PPLI 



M4SRC 

Mode IV Source 

0 = Information Unavailable 

1 = UPX 29 

2 = Link 

3 = Manual 

4 = CEC 

5 - ecu 

6 = Linkl6PPi’ % '« 

._ 

1 

IDSRC 

ID'"'''ur-v 7^ L t \ \ 

’ 1 1 
j. ; !IV 4 1 "I 1 , 

^ ^ ^ "h. 

-> "is ^ ' 

% '% % 

^ M' 1 ^ 

1 %or> Lion^”' 

r#i='CT:p 

12 = CECU 

TT 

Trouble track 

0 = Not a trouble track 

1 = Trouble track 



PLAT 

Platform 



ACT 

Activity 



SPECTYP 

Specific type 



CGTN 


Additional 

track 

numbers 


CEPN 




CECUID 
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ANNEX K-C Data Extraction Points 

Use this annex to provide detailed data extraction points for each 
system in the test. 

ANNEX K-D Security Classification Guides 

This annex should contain the security guidelines for the test. 
These include specific security instructions for the handling of the data 
to be encountered during the test. 
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APPENDIX L: SAMPLE FORM 


Table L-1. Test Observation Report (TOR) 


Test Observation Report (TOR) 

Classification: 

(circle one) 

UNCLAS 

CONF 

SECRET 

System(s) TOR is 
written against: 

TOR Number: 

Operator Position: 

Reported by: 

Phone #: 

Email: 

Date of event: 

Time: 

Zulu Time: 

Or 

Local Time: 

Tape Numbers: 

Description: 

Impact: (optional) 

TOR Instructions 

Classification 

Security classification of the TOR. . 

System 

Aircraft, ship, or land based site (TAOC, CRC, ICC, DDG, etc.} affected 
by observed anomaly. 

TOR number 

TOR number (to be assigned when entered into tracking table or 
database) 

Operator Position 

Watch/test station where the observation was made. 

Reported bv 

Originator of the TOR and command. 

Phone Number 

Phone number originator can be reached at after event. 

Tape Numbers 

Complete tape number for the DX tape to use for analysis (include 
system, if knovm). 

Date of event 

Date of observation (MMDD). 

Time 

Time of observation. Designate either Zulu or Local Time. 

Description 

A thorough description of the observation. Should include system 
name and configuration, scenario information, tracks, identifications, 
track kinematics, and other information necessary to establish the 
same environment as the observation. Also Include information as to 
what actually happened during the observation. 

Impact 

A brief description of the operator Impacts this deficiency had on the 
operator or system if not corrected. 
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APPENDIX M: POINTS OF CONTACT 

Identify names of participants and their roles in the event. Provide 
contact information. 


Table M-1. Participants in the JDEP Planning 




Last name, First Name 

Company, Office 
Symbol 




















Table M-2. Test Directors/Site Test Directors 





m ■ 

_ 


For example; "Test 
Director (Primary)" 




For example: "NAWC-VtD 
(E2C)" 




For example: "Data 
Distribution Manager" 




For example: "Data 
Collection Manager" 





Table M-3. Data Collection Team 





<■ 



For example: 
"REPOSITORY" 

For 

example: 
"NAVSEA 
Corona, CA" 

For example; "DX 
Coordinator, NAVSEA 
Corona" 











Table M-4. Site Leads/POCs 


Site 

Friniary/ 

Attemate 

SitePOC 

_ - - 

Phone 

&MaU 

For example, 
"NAWC-AD (E- 
20" 

Primary 




Alternate 
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Table M-5. Lead Analysts 
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Fitness Report of ELSTAD 
12/21/04 11:32 AM 





































































































































































Ghvzel Paul LCDR SIAP-AN 


From: 

Youmans, Betty [eyoumans@spa.coml 

Col, 

Sent: 

Tuesday, December 21, 2004 12:36 PM 

To: 

Ghyzel Paul LCDR SIAP-AN 

f~/j Uc/iA/j Ac, f 

Cc: 

Karoly Steve Civ SIAP-AN 

A -k 

dl ,-iACe/y'^cf^T7. 

Subiect: 

RE: Documents on Worksite 

Paul, 




rf 


Dutch is looking for two things, a checklist in the Exec Summary that a TRR must address 
and a mention, of accreditation in the List in Chapter 6. 


I have updated the Exec Summary Paragraph to read: 

In the Executive Summary, provide a summary of essential information regarding the 
testing/siraulation event. Include high-level objectives, dates and location of the event 
and how the results will be used. Provide a summary to support a recommendation whether 
or not to proceed forward with the test based on the following outline: 


1. System test status and checkout performance 

2. FOM status (M&S venues) 

3. Equipment and computer program configuration 

4. Test coordination 

5. Success criteria 

6. Go/No-Go criteria 

7. Recommendation for accreditation of federation (M&S venues) 

8. Recommendation whether or not to proceed with test as planned. 

I have augmented the list in Chapter 6 to read the following (added bottom 2 entries): 


1. System test status and checkout performance 

2. FOM status (M&S venues) 

3. Equipment and computer program configuration 

4. Test objective(s) and procedure review 

5. Test coordination 

6. Security 

7. Success criteria 

8. Go/No-Go criteria 

9. Real-time data requirements to include format, 

10. Quick-look data requirements to include format 

available) . , , , ^ c • 

11. Final data requirements to include format, algorithms, and data definitions 

12. Recommendation for accreditation of federation (M&S venues) 

13. Recommendation whether or not to proceed with test as planned. 


algorithms, and data definitions 
algorithms, and data definitions (if 


I believe these should meet Dutch's comments. Do you agree, or do you want any further 
changes? If no further changes, then I will send back the updated version with Kelly 
this afternoon. I will just slip in the two new sets of pages impacted. 


Betty 


-Original Message- 

From: Ghyzel Paul LCDR SIAP-AN [mailto:Paul 

Sent: Tuesday, December 21, 2004 7:56 AM 

To: Youmans, Betty 

Cc: Karoly Steve Civ SIAP-AN 

Subject: FW: Documents on Worksite 


Betty, 


.Ghyzel@Siap.Pentagon.mil] 




The std test plan and test report templates are signed, but I need you to come get the std 

1 





